"Express Hail" Mailing Label Number: EL530408928US 



Date of Deposit: April 27, 200S . ;jc777 U S PTO 

II III I III 

04/27/00 



IIIIIIDIIIIIIMIIIII »§i 

Postal Service "Express Mail Post Office to flA/OT /flfl =) i2 L 

Addressee" service under 37 C.F.R. 1.10 on, II*// / I WW S 

the date indicated above and is addressed/to the 
Assistant Comm/ssioner For Patents^-Wasl 
D.C. 20231 



flA,Mk £( Jit 

i/rrella ' / 



Attorney's Docket No. XER 2 0341 
D/A0452 

UTILITY PATENT APPLICATION TRANSMITTAL LETTER 

(Only for nonprovisional applications under 37 CFR 1.53(b) 

Assistant Commissioner for Patents 
Washington, D.C. 20231 

Transmitted herewith for filing in the patent application of: 

Robert P. Loce 
Beilei Xu 

Entitled: METHOD AND SYSTEM TO ADJUST FOR IMAGE ERRORS INDUCED BY A LENS ARRAY 
Enclosed are: 

[X] 10 sheets of specification, 4 sheets of claims, 1 sheet of Abstract. 

[X] 3 sheets of drawings, Figures 1-3. 

[X] an assignment of an invention to XEROX CORPORATION (and transmittal 

therefor) . 

[X] a Declaration and Power of Attorney for Patent Application. 

[] Filing under C.F.R. 1.41(c) 

[ ] Information Disclosure Statement and PTO-1449 

[ ] Preliminary Amendment 



CLAIMS AS FILED 


FOR 


NO. 


FILED 


NO. EXTRA 


RATE 


FEE 


TOTAL CLAIMS 


18 


- 20 = 


0 


[XX] 
[ ] 


$18 LARGE 
$09 SMALL 




$ o 


INDEPENDENT 
CLAIMS 


4 


- 3 = 


1 


[XX] 
t ] 


$78 LARGE 
$39 SMALL 




$ 78.00 


BASIC FEE [XX] 


LARGE 


ENTITY 


$690 [ 


] 


SMALL ENTITY 


$345 










TOTAL 


FILING FEE 


? 768 



lh 

□ 



Please charge Deposit Account No. 24-0037 in the amount of $ 768.00 . 

The Commissioner is hereby authorized to charge any additional fees which 
may be required at any time during the prosecution of this application 
without specific authorization, or credit any overpayment to Deposit Account 
No. 24-0037. 

A check in the amount of $ to cover the filing fee is enclosed. 



GO 



Attorney of Record 
W. Scott Harder s 
Reg. No. 42,629 
FAY, SHARPE, FAGAN, 
MINNICH & McKEE, LLP 
1100 Superior Avenue 
Seventh Floor 

Cleveland, Ohio 44114-2518 
(216) 861-5582 



XER 2 0341 
D/A0452 



METHOD AND SYSTEM TO ADJUST FOR IMAGE ERRORS INDUCED 
BY A LENS ARRAY 

Background of the Invention 

This invention generally relates to the art of digital imaging. The 
invention finds particular application in adjusting for image errors or artifacts 
induced by a lens array employed to either scan physical images into an image 
5 representation, or to output image representations onto physical media. It is to be 
appreciated however, that the present invention finds further application in other 
imaging devices with spatially varying error or artifact functions. 

Image acquisition devices, such as scanners, and image output 
devices such as printers or copying machines, typically employ a lens mechanism to 
10 either transfer a physical image onto a photo sensitive surface in the case of an 
image input or scanning device, or transfer a digital image representation onto a hard 
copy or physical output media. 

Indeed, certain optical systems used in image reading or writing 
possess a spatially varying point-spread-function (PSF). An example is the local 
15 blurring and image doubling that occur due to tilted rods in a Selfoc® lens array. 
This local PSF effect can cause severe streak artifacts in scanned and printed 
images, particularly halftone images. In many input scanners, scan bars are rejected 
due to this problem. This defect and other array lens defects vary across the field, 
hence, a restoration or pre-compensation based on the traditional assumption of a 
20 spatially uniform PSF do not yield satisfactory results. 
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The present invention contemplates an improved method and 
apparatus to correct or adjust for errors in an optical system using image post- 
acquisition and pre-output techniques which overcome the above-referenced 
problems and others. 

5 

Summary of the Invention 

In accordance with one embodiment of the present invention, a 
method of altering an image representation to adjust for artifacts attributable to an 
array lens include obtaining a characterization for the array lens. From the 
10 characterization, compensation parameters are determined for a plurality of locations 
across the array lens and these determined parameters are stored. 

In accordance with another aspect of the present invention, the 
artifacts attributable to the array lens are induced during image scan using the array 
lens. This results in an electronic image representation which contains the artifacts. 
15 The method further includes applying the compensation parameters to the electronic 
image representation including the artifacts, which results in a post-compensated 
electronic image representation. 

In accordance with another aspect of the present invention, the 
artifacts attributable to the array lens are induced during image output using the 
20 array lens. The method further includes applying the compensation parameters to an 
electronic image representation without the artifacts, or a desired electronic image 
representation. This results in a pre-compensated electronic image representation. 

In accordance with another aspect of the present invention, the 
characterization step includes measuring optical performance of the array lens at a 
25 plurality of locations across the array lens and/or estimating optical performance of 
the array lens at a plurality of locations across the lens. 

In accordance with another aspect of the present invention, the 
method further includes applying the compensation parameters to the image 
representation with an iterative restoration method such as the Maximum 
30 Likelihood-Expectation Maximization method (ML-EM), sharpening filters, 
windowed-wiener spectrum technique, and spatial convolution. 
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In accordance with another embodiment of the present invention, an 
imaging apparatus includes a plurality of light sources and an array lens which 
focuses emitted light onto a desired receptor. A memory is also included which 
stores a plurality of parameters used to compensate for artifacts which are induced 
5 by the array lens, and a processor which applies the compensation parameters to the 
image representation resulting in a compensated image representation. 

In accordance with another aspect of the present invention, the 
imaging apparatus employs the array lens to acquire an image representation from a 
physical image. Thus, artifacts are induced in the image representation to which a 
10 processor applies the compensation parameters resulting in a post-compensated 
image representation. 

In accordance with another aspect of the present invention, the 
imaging apparatus employs the array lens to produce a physical image from a 
desired image representation. The processor applies the compensation parameters to 
15 the desired image representation which results in a pre-compensated image 
representation suitable for output by an apparatus including the array lens. 

In accordance with another aspect of the present invention, the array 
lens includes a plurality of adjacent rods arranged in a one-dimensional array and/or 
a plurality of adjacent rods arranged in a two-dimensional array. 
20 In accordance with another embodiment of the present invention, a 

digital imaging method includes determining an error attributable to at least one 
selected coordinate on an array lens. A physical image is then scanned using the 
array lens with the determined error, which results in an image representation 
including the error or artifacts. The determined error in the scanned physical image 
25 is then compensated for which results in a post-compensated image representation. 

In accordance with another aspect of the present invention, the 
determining an error step includes measuring errors induced by the array lens at 
selected locations relative to the array lens. 

In accordance with another aspect of the present invention, the 
30 compensating step includes altering the image representation to adjust spatially 
varying errors induced by the array lens. 



In accordance with another embodiment of the present invention, a 
digital imaging method includes determining an error attributable to at least one 
selected coordinate on an array lens. A desired image representation is then 
received and the determined error is compensated for resulting in a pre-compensated 

5 image representation. The pre-compensated image representation is then output 
onto a physical media. 

In accordance with another aspect of this embodiment of the present 
invention, the determining an error step includes measuring errors induced by the 
array lens at selected locations relative to the array lens. 

10 In accordance with another aspect of this embodiment of the present 

invention, the compensating step includes altering the image representation to adjust 
for spatially varying errors induced by the array lens. 

One advantage of the present invention resides in the correction of 
errors induced by an array lens during image acquisition and/or image output. 

15 Another aspect of the present invention resides in the ability to 

restore scanned images, and/or pre-compensate for output images according to 
errors induced by an array lens, or other spatially varying neighborhood effects. 

Still other advantages of the present invention will become apparent 
to those of ordinary skill in the art upon reading and understanding the following 

20 detailed description of the preferred embodiments. 

Brief Description of the Drawings 

The invention may take physical form in certain parts and 
arrangements of parts, and in certain steps and arrangements of steps. The drawings 
25 are only for purposes of illustrating the preferred embodiments and are not to be 
construed as limiting the invention. 

FIGURE 1 is a functional block diagram of an imaging apparatus 
suitable to practice an embodiment of the present invention; 

FIGURE 2 is a side elevational view of a one-dimensional array lens; 

30 and, 

FIGURE 3 is is a functional block diagram of an apparatus suitable to 
practice another embodiment of the present invention. 



Detailed Description of the Preferred Embodiments 

With reference to Figure 1, an imaging apparatus is illustrated which 
compensates for artifacts or errors introduced during the acquisition of a physical 
5 document 12 into electronic format. An exemplary method to acquire or convert a 
physical image or hard copy 12 to a digital image representation is to employ a 
scanner 14 or the like. Typically, scanner 14 includes an array lens, conventional 
optics, or other light focusing devices. Because the optics employed in scanners are 
typically imperfect, errors or artifacts are induced during conversion or acquisition 

10 of the physical image 12 as a digital image representation 16. 

An exemplary array lens 20 is illustrated in Figure 2. The array lens 
includes a plurality of small rods 22 disposed adjacent to one another in a line. 
Typically, the rods 22 are formed from glass or fiber optic cable and the like. 
During use, the lens is used to focus or direct light 24 from one side of the lens 26 to 

15 the other 28. Particular attention to individual rods 22f and 22k illustrate an 
imperfection in the array 20. That is, at times the individual rods are not disposed in 
a perpendicular arrangement within the lens and can cause light 24 to appear 
transposed on the output side 28 of the array lens. This artifact or error is best 
illustrated by dashed line 30 indicating the path a particular light beam travels 

20 through the array. While the array lens 20 illustrated includes a single row of rods 
those skilled in the art will recognize that typical "one-dimensional" arrays often 
employ several rows of closely spaced rods. The "one-dimensional" name refers 
here to the object plane or target area intended to be observed or imaged with the 
lens. Similarly, but not illustrated, a "two-dimensional" array employs multiple 

25 rows of rods, with the "two-dimensional" name referring to the two-dimensional 
object plane or target area to be observed or imaged. 

Referring back to Figure 1, the image representation containing 
artifacts 16 is forwarded from the scanner 14 to an image processor 36. As will be 
more thoroughly discussed below, image processor 36 generally proceeds to a 

30 specified image location, for example a target pixel within the image representation, 
and then performs a restoration operation. The restoration operation for any 
particular pixel is retrieved from memory 38 and, when applied to the image 



representation 16 yields an estimate of the image as if it were scanned with idealized 
optics producing a nominal uniform error such as a preferred point spread function. 
The restoration operation employs neighboring pixels about the target pixel in 
determining the restored value. Following the compensation by the processor 36, a 
post-compensated digital image representation 40 is output for further manipulation, 
storage, or distribution or the like as exemplified by the illustrated devices 42. 

Memory 38 is loaded with compensation parameters determined, for 
example, by performing a lens characterization or performance measurement 44 on 
the array lens 20. An exemplary lens characterization employs classical modulation 
transfer function (MTF) measurement and/or estimation techniques sampled across 
the field of the array lens. For example, image scans may be obtained of an edge, 
line, point, and/or a periodic pattern. Alternately, whole field characterizations may 
be accomplished using speckle techniques, and/or periodic objects such as "on and 
off' lines at different spatial frequencies. Moreover, those skilled in the art will 
appreciate that the lens characterization 44 can be accomplished in a variety of 
locations, such as in an assembly plant prior to delivery to a customer, in a 
refurbishing shop, or in the field, i.e. at a user's location. 

From the characterization 44, parameters are determined 46 
according to the specific compensation method in use by the image processor 36. 
For example, point spread function (PSF) parameters at various locations along the 
array lens are extracted for the Maximum Likelihood-Expectation Maximization 
(ML-EM) method, Modulation Transfer Function (MTF) parameters are extracted 
for the windowed wiener spectrum method, and a wide variety of sharpening filter 
coefficients are extractable such as linear, non-linear and the like. Once these 
parameters are determined, they are forwarded to the memory 38 for access by the 
image processor as discussed above. 

Specifically, a digital image F" digitized by an input scanner can be 
represented as an ideal digital image F that has been imaged by a "convolution" 
process PSF(ij;i,j), where PSF denotes the Point Spread Function of an imaging 
system that can vary with location (ij): 

F\iJ) = £ - /, / - j)PSF(i', f; i, j) (l) 



For an ideal scanner, PSF is constant across the imaging field, i.e. PSF(i,j; 
i,j) = PSFideai(i,j) for all /' and j. In this case, conventional image restoration 
techniques, such as Wiener filtering technique, can be suitably applied to estimate 
the original documents from the scanned images. However, if there are defects in 
the system, such as a tilted rod in a Selfoc® lens array, PSF(iJ;iJ) may vary with / 
and/ In general, PSF(i,j;i,j) can be represented by two components within another 
nonlocal "convolution" process as shown by Eq. (2): 

PSFQJJJ) = ^PSF^ii'-iJ'-fiPSF^i'J'JJ) (2) 

>'•/ 

where PSF,d ea i is the uniform component and PSF SV denotes a spatially varying 
component. 

Those skilled in the at will appreciate that the techniques more fully 
explained below is capable of correcting for both of the effects simultaneously. 
Note that the variation in PSF SV due to defects in a Selfoc® lens array typically 
appears only in the fast scan direction (/). Hence, in the typical scenario of interest 
PSF can be rewritten as 

PSF(iJ;i,j) = PSF(iJ;i) (3) 

To better understand the spatially varying nature of this nonlogical 
"convolution" process, consider an example where PSF SV can be characterized by a 
Gaussian distribution. For this example, according to Eq. (3), the standard deviation 
of the Gaussian varies with the fast scan location of the distribution (location /). The 
equations below show the relationship between the actual scanned image F", ideal 
scanned image F\ , and component point spread functions. Equation (4) shows the 
nonlocal "convolution" process relating F " with F'. 

F\iJ) = ZF'd'-hf-APSF^i'J'-J) ( 4 ) 

Equation (5) shows the conventional convolution process relating F' with F". 

F'(i,j) = F(i,j)®PSF ideal Q,j) (5) 



Artisans will recognize that other defects might also yield spatially varying PSF. 
For example, when the scanning bars are not parallel to the platen surface or when 
an object is not placed in close contact with the scanning surface (e.g. bound book, 
wrinkled sheets, and 3D objects), PSF also varies with spatial location. When a 
spatially varying PSF is present, a correction method based on a uniform PSF might 
produce images with artifacts due to over-or under-corrections. 

As an example of a specific lens characterization and compensation 
parameter determination, a spatially varying PSF is obtained as follows. We assume 
the spatially varying PSF of a system is generally constant over time or varying at 
such a rate that a sensing or re-calibration can be suitably performed. 

There are two general approaches to obtain the spatially varying PSF: 
the use of an imaging model of the system to predict the PSF at a point; or a 
calibration procedure. Alternately, a combination of a calibration procedure and an 
imaging model can be employed. The following image measurement-based method 
is exemplary: 

Position a fine line in the object plane of an array lens (such as those 
manufactured under the trademark 'Selfoc') scanner perpendicular to the 
array length; 

Scan the image plane with a slit detector to obtain the local line spread 
function; 

Repeat the above process for all positions of interest in the imaging field; 
Rotate the line and slit in the opposite direction, and repeat the above 
process. 

If an effect is periodic, such as the nominal variation along the array 
lens when it is defocused, one can acquire data for one period of the effect and use 
it when operating on all periods. 

The general methodology at this step is that an algorithm would first 
proceed to an image location (target pixel). Then a restoration operation suitable for 
that location is performed. The restoration yields an estimate of the image as if it 
was scanned with idealized optics producing a nominal uniform preferred PSF. The 
restoration operation employs neighboring pixels about that target pixel in 
determining the restored value of the pixel. For example, the operation could be like 



a "local convolution," or other local restoration method. Equation (6) shows this 
operation abstractly. 

F'(iJ) = F(i,j)® i RES(iJ;i) (6) 
where F\i,j) is the estimate of the image at point (ij) as if it were scanned with 
5 idealized optics producing nominal uniform preferred PSF; ® 1 is a local 
neighborhood operation about pixel /' (similar to a local convolution); and RES (/', j; 
/') is a restoration kernel that contains processing coefficients or information for 
restoration about pixel i. Below, we show a specific local restoration process which 
effectively corrects image doubling artifacts like those encountered with Selfoc® 
10 tilted rods. 



Iterative ML-EM Restoration Method 

To compensate for the spatially varying PSF, it is desirable to 
develop an image restoration technique that can directly incorporate this PSF into its 
15 operation. Also, the method should work well in an image doubling setting, such as 
one might encounter with array lens tilted rods. One such method is the Maximum 
Likelihood-Expectation Maximization method (ML-EM) as shown by 

t^T ; F^\m-t,n- S )PSF sv (m,ri;i) K) 

20 

where F' (k + l) is the estimate at the (k+1) iteration, and the operation is being 
performed about pixel / as seen in the PSF SV distribution. 

With reference now to Figure 3, an imaging apparatus is shown to 
pre-compensate a digital image representation prior to outputting where the output 
25 engine employs non-ideal optics. A digital image representation 50 is created or 
retrieved from a digital image source 52. At this point the image representation is 
error free, that is, it does not include artifacts. Image processor 54 applies pre- 
compensation parameters to the desired digital image representation 50 to produce a 
pre-compensated image representation 56. In other words, the image processor 54 



applies an inverse of the errors or artifacts expected to be induced later during image 
output. The pre-compensated image representation 56 is then forwarded to an 
output engine 58 which includes non-ideal optics such as a lens array 20 (Figure 2). 
The physical output 60 thus appears to have been printed on an ideal or error free 
5 output terminal. 

Those skilled in the art will appreciate that in some instances pre- 
compensation processing occurring in processor 54 will require producing or 
applying negative values to the pre-compensated image representation 50. These 
negative values can be implemented in an offset gain system, or alternately they may 

10 be truncated with the errors diffused locally via an error diffusion method. 

The invention has been described with reference to the preferred 
embodiments. Modifications and alterations will naturally occur to others upon 
reading and understanding the preceding detailed description. It is intended that the 
invention be construed to include all such modifications and alterations insofar as 

15 they come within the scope of the appended claims or the equivalents thereof. 
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Having thus described the invention, we claim: 

A method of altering an image representation to adjust for artifacts 
attributable to an array lens, the method comprising: 

obtaining a characterization at selected locations across the array lens; 
from the characterization, determining compensation parameters for a 
5 plurality of locations across the array lens; and 

storing the determined compensation parameters. 

2. The method as set forth in claim 1, wherein the artifacts attributable to the 
array lens are induced during image scan using the array lens, resulting in an electronic 
image representation including the artifacts, the method further comprising: 

applying the compensation parameters to the electronic image 
5 representation including the artifacts, resulting in a post-compensated electronic image 
representation. 

3. The method as set forth in claim 1, wherein the artifacts attributable to the 
array lens are induced during image output using the array lens, the method further 
comprising: 

applying the compensation parameters to an electronic image 
5 representation without the artifacts, resulting in a pre-compensated electronic image 
representation. 

4. The method as set forth in claim 1, wherein the obtaining a 
characterization step comprises: 

measuring optical performance of the array lens at a plurality of locations 
across the array lens. 

5. The method as set forth in claim 1, wherein the obtaining a 
characterization step comprises: 
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estimating optical performance of the array lens at a plurality of locations 
across the array lens. 



6. The method as set forth in claim 1, wherein the obtaining a 
characterization step comprises: 

measuring optical performance of the array lens at at least one location on 
the array lens; and 

estimating optical performance of other locations on the array lens based 
on the measurements. 

7. The method as set forth in claim 1, further comprising: 

applying the compensation parameters to the image representation with an 
iterative restoration method selected from the set of ML-EM method, sharpening filters, 
windowed-wiener spectrum technique and spatial convolution. 



desired receptor, the array lens inducing artifacts in an image representation on the 
5 receptor; 

a memory which stores a plurality of parameters to compensate for the 
array lens induced artifacts; and 

a processor which applies the compensation parameters, resulting in a 
compensated image representation. 

9. The imaging apparatus as set forth in claim 8, wherein: 

the imaging apparatus employs the array lens to acquire an image 
representation from a physical image, thereby inducing artifacts in the image 
representation, and 




An imaging apparatus comprising: 
at least one light source; 

an array lens which focuses emitted light from the light source onto a 
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the processor applies the compensation parameters to the image 
representation including the artifacts, resulting in a post-compensated image 
representation. 



10. The imaging apparatus as set forth in claim 8, wherein: 

the imaging apparatus employs the array lens to produce a physical image 
from a desired image representation, and 

the processor applies the compensation parameters to the desired image 
representation, resulting in a pre-compensated image representation. 

11. The imaging apparatus as set forth in claim 8, wherein the array lens 
comprises a plurality of adjacent rods arranged in a one-dimensional array. 

12. The imaging apparatus as set forth in claim 8, wherein the array lens 
comprises a plurality of adjacent rods arranged in a two-dimensional array. 



array lens; 

scanning a physical image using the array lens with the determined error, 
resulting in an image representation including artifacts; and 

compensating for the determined error in the scanned physical image, 
resulting in a post-compensated image representation. 

14. The digital imaging method as set forth in claim 13, wherein the 
determining an error step comprises: 

measuring errors induced by the array lens at selected locations relative to 

the array lens. 

15. The digital imaging method as set forth in claim 13, wherein the 
compensating step comprises: 




A digital imaging method comprising: 

determining an error attributable to at least one selected coordinate on an 
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altering the image representation to adjust for spatially varying errors 
induced by the array lens. 




A digital imaging method comprising: 

determining an error attributable to at least one selected coordinate on an 



array lens; 



5 



receiving a desired image representation; 

compensating for the determined error in the image representation 



resulting in a pre-compen sated image representation; and 

outputting the pre-compen sated image representation on a physical media. 

17. The digital imaging method as set forth in claim 16, wherein the 
determining an error step comprises: 

measuring errors induced by the array lens at selected locations relative to 

the array lens. 

18. The digital imaging method as set forth in claim 16, wherein the 
compensating step comprises: 

altering the image representation to adjust for spatially varying errors 
induced by the array lens. 
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METHOD AND SYSTEM TO ADJUST FOR IMAGE ERRORS INDUCED BY A 
LENS ARRAY 



Abstract of the Disclosure 

An imaging apparatus includes an image acquisition section which 
converts a physical image to a digital image representation using non-ideal optics such as 
an array lens. This image acquisition results in a digital image representation containing 
5 errors or artifacts. A memory in data communication with a processor stores a plurality 
of compensation parameters selected for use in correcting errors induced by the lens 
array. The compensation parameters are determined by performing a lens 
characterization which includes measuring lens performance at a plurality of locations 
along the lens. After the processor adjusts the image representation, the post- 
10 compensated digital image representation may be further processed, stored, transferred, 
and the like. According to another embodiment of the invention, a non-ideal array lens 
induces errors in an image representation during a printing or output operation. 
Similarly, an image processor applies pre-compensation parameters to the desired or ideal 
image representation in electronic form to compensate for errors which are known to be 
15 induced by the lens during the output operation. Accordingly, when the pre-compensated 
image representation is output using the non-ideal lens, the physical image output appears 
to have been printed with an ideal lens. 
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SEND CORRESPONDENCE TO: 
Albert P- Sharpe. ill, Esq. 
Fay, Sharpc, Fagan, 
Minnieh & McKee, LLP 
1 l(X> Superior Avenue, 7th Floor 
Cleveland, OH 441X4-2518 



DIRECT TELEPHONE CALLS TO: 
(name and telephone number) 



Albert P. Sharpe, 111 
(216) 86J-SS82 



I hereby declare that all statements made herein of my own knowledge are true and that all statements made 
on information and belief are believed to be true; and further that these statements were made with the 
knowledge thai willful false statements and the like so made are pumslutble by fine or imprisonment, or both 
under Section 1001 of Tide 18 of the United States code and that such willful false statements may jeopardize 
the vahdiry of the application or any patent issuing thereon. 



Full name of sole or first joint inventor: Robert P. Loce 

Inventor's* Signature:. 
Residence Webster, New York 

Country of Citizenship: U.S.A. Post Office Address: 736 Hailey Drive 

Webster, NY 1458Q 



Full name of second joint inventor: Beilei Xu 



Inventor's Signature:,; 
Residence: Penfield, New York 
Countty of Citizenship: P. R. CHINA 



Date: 

Post Office Address: 8 Pipers Meadow 
Trad 

Penfield, NY 14526 



